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This paper explores how curriculum and classroom conceptual and procedural emphases affect the 
learning of algebra for students of color. Using data from a longitudinal study of the Connected 
Mathematics Program (CMP), we apply cross-sectional HLM to lend explanatory power to the 
longitudinal analysis afforded by Growth Curve Modeling that we have reported elsewhere. Overall, we 
find that the achievement gaps tend to decrease over the course of the middle grades. However, differences 
in open-ended problem solving and equation solving performance persist for African American students. 
Classroom conceptual and procedural emphases also appear to differentially influence the performance of 
Hispanic and African American students, depending on the aspect of algebra learning that is being 
measured. 
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Classrooms in the United States are becoming increasingly ethnically diverse. However, disparities in 
the mathematics achievement of different ethnic groups remain a persistent challenge (Lubienski & 
Crockett, 2007). Since teaching and learning are cultural activities, students with different ethnic and 
cultural backgrounds may respond differently to the same curriculum. Given the development and 
implementation of curricula based on the Standards documents developed by the National Council of 
Teachers of Mathematics (NCTM, 1989, 2000), a key question about curriculum reform is: How does the 
use of a Standards-based curriculum like CMP impact the learning of students of color as compared to 
White students? 

In our project, Longitudinal Investigation of the Effect of Curriculum on Algebra Learning (LieCal), 
we used a longitudinal design to examine the similarities and differences between a Standards-based 
curriculum called the Connected Mathematics Program (CMP), and more traditional curricula (non-CMP). 
We investigated not only the ways and circumstances under which the CMP and non-CMP curricula 
affected student achievement gains, but also the characteristics of these reform and traditional curricula 
that hindered or contributed to the gains. One aspect of the LieCal analysis was an examination of 
potentially differential effects of curriculum and procedural and conceptual emphases in the classroom on 
the achievement of students of color. In this paper, we present results from a cross-sectional analysis of 
student growth within each grade level. This analysis allows us to add depth to our previous analysis using 
Growth Curve Modeling by probing effects that are significant at individual grades but which were not 
uncovered in our longitudinal analysis. 


Background 


Algebra readiness has been characterized as the most important “gatekeeper” in school mathematics 
(Pelavin & Kane, 1990). In particular, success in algebra and geometry has been shown to help narrow the 
disparity between minority and non-minority participation in post-secondary opportunities (Loveless, 
2008). Research shows that completion of an Algebra II course correlates significantly with success in 
college and with earnings from employment. The National Mathematics Advisory Panel (2008) found that 
students who complete Algebra II are more than twice as likely to graduate from college as students with 
less mathematical preparation. Furthermore, the African-American and Hispanic students who complete 
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Algebra II cut the gap between their college graduation rate and that of the general student population in 
half. However, success in high school algebra is dependent upon mathematics experiences in the middle 
grades. In fact, middle school is a critical turning point for students’ development of algebraic thinking 
(College Board, 2000). 

In a Standards-based curriculum like CMP, the focus is on conceptual understanding and problem 
solving rather than on procedural knowledge. Students are expected to learn algorithms and master basic 
skills as they engage in explorations of worthwhile problems. However, a persistent concern about 
Standards-based curricula is that the development of students’ higher-order thinking skills comes at the 
expense of fluency in computational procedures and symbolic manipulation. In addition, it is not clear 
whether this potential trade-off might play out differently for students from different ethnic backgrounds. 
Some reports have suggested that Hispanic and African American students using the CMP curriculum may 
in fact show greater achievement gains than students from other backgrounds (Rivette, Grant, Ludema, & 
Rickard, 2003). Still, research is needed to assess whether and how the use of a Standards-based 
curriculum such as CMP can improve the mathematics achievement of all students while helping to close 
achievement gaps (Lubienski & Gutiérrez, 2008; Schoenfeld, 2002). 

Since the effectiveness of a curriculum depends critically on how it is implemented by teachers in real 
classrooms, studies of the effectiveness of Standards-based curricula must examine how teachers use the 
curricula (Kilpatrick, 2003; NRC, 2004; Wilson & Floden, 2001). The data gathered must be analyzed in 
appropriate ways to control for variations in classroom instruction and the learning environment. In order 
to determine the effects of curriculum on learning, it is essential to examine the classroom experiences of 
the teachers and students who are using the different curricula. In this paper, we take features of classroom 
instruction into consideration when we examine the impact of curricula on students’ learning of algebra. In 
particular, we examine the extent to which teachers emphasize concepts and procedures in the classroom. 
As was reported by Moyer, Cai, Nie, and Wang (2011), CMP teachers placed more emphasis on 
conceptual understanding whereas non-CMP teachers placed more emphasis on procedural knowledge. 

Our previous longitudinal analyses of the LieCal data using Growth Curve Modeling showed that over 
the three middle school years, African American students experienced greater gain in symbol manipulation 
when they used a traditional curriculum. The use of either the CMP or a non-CMP curriculum improved 
the mathematics achievement of all students, including students of color. The use of CMP contributed to 
significantly higher problem-solving growth for all ethnic groups (Cai, Wang, Moyer, Wang, & Nie, 
2011). In this paper, we take a cross-sectional approach and examine the achievement of students of color 
in each grade level while controlling for the conceptual and procedural emphases in classroom instruction. 


Method 
Sample 


The LieCal project was conducted in 14 middle schools of an urban school district serving a diverse 
student population. When the project began, 27 of the 51 middle schools in the district had adopted the 
CMP curriculum, and the remaining 24 had adopted more traditional curricula. Seven schools were 
randomly selected from the 27 schools that had adopted the CMP curriculum. After the seven CMP 
schools were selected, seven non-CMP schools were chosen based on comparable demographics. In sixth 
grade, 695 CMP students in 25 classes and 589 non-CMP students in 22 classes participated in the study. 
We followed these 1,284 students as they progressed from grades 6 to 8. Approximately 85% of the 
participants were minority students: 64% African American, 16% Hispanic, 4% Asian, and 1% Native 
American. Male and female students were almost evenly distributed. 


Assessing Students’ Learning 


Learning algebra involves honing procedural skills with computation and equation-solving, fostering a 
deep understanding of fundamental algebraic concepts and the connections between them, and developing 
the ability to use algebra to solve problems. Thus, to assess students’ learning of algebra, it is important to 
consider their conceptual understanding, their symbol manipulation skills, and their ability to solve 
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problems. We used state test scores in mathematics and reading as measures of prior achievement. We 
used LieCal-developed multiple-choice and open-ended assessment tests as dependent measures of 
procedural knowledge and conceptual understanding in algebra, respectively. The two LieCal-developed 
tests were administered four times, once as a baseline in the fall of 2005, and again each spring (2006, 
2007, and 2008). 

We used multiple-choice items to assess whether students had learned the basic knowledge required to 
perform competently in introductory algebra. Each version of the multiple-choice test was comprised of 32 
questions that assessed five mathematics components (Mayer, 1987): translation, integration, planning, 
computation (or execution), and equation solving. For this paper, we report on the results from the 
translation, computation, and equation-solving components of the multiple-choice tasks. In addition, the 
open-ended tasks were designed to assess students’ conceptual understanding and problem-solving skills. 
These tasks were adopted from various projects including Balanced Assessment, the QUASAR Project 
(Lane et al., 1995), and a cross-national study (Cai, 2000). Since only a small number of open-ended tasks 
can be administered in a testing period, and since grading students’ responses to such items is labor- 
intensive, we distributed the non-baseline tasks over three forms (five items in each form) and used a 
matrix sampling design to administer them. Examples of the items and tasks used in the LieCal 
assessments can be found in Cai et al. (2011). 

The multiple-choice items then were scored electronically, either right or wrong. The open-ended tasks 
were scored by middle school mathematics teachers, who were trained using previously developed holistic 
scoring rubrics. Two teachers scored each response. On average, perfect agreement between each pair of 
raters was nearly 80%, and agreement within one point difference out of 6 points (on average) was over 
95% across tasks. Differences in scoring were arbitrated through discussion. The two-parameter Item 
Response Theory (IRT) model was used to scale student assessment data on both multiple-choice tasks and 
open-ended tasks (Hambleton, Swaminathan, & Rogers, 1991; Lord, 1980). 


Conceptual and Procedural Emphases as Classroom-level Variables 


Mathematical proficiency includes both conceptual and procedural aspects (NRC, 2001), and teachers 
can shape instruction in ways that emphasize either or both aspects. We used conceptual and procedural 
emphases as classroom variables when examining the impact of curriculum on students’ learning. To do 
so, we estimated the levels of conceptual and procedural emphases in the CMP and non-CMP classrooms 
using data from 620 lesson observations of the LieCal teachers, which we conducted while the students 
were in grades 6, 7, and 8. Each class was observed four times, during two consecutive lessons in the fall 
and two in the spring. Further details about the observations are documented in Moyer et al. (2011). One 
component of the observation was a set of 21 items using a 5-point Likert scale to rate the nature of 
instruction for each lesson. Of the 21 items, four were designed to assess the extent to which a teacher’s 
lesson had a conceptual emphasis. For example, observers rated a lesson’s conceptual emphasis using the 
following item: “The teacher’s questioning strategies were likely to enhance the development of student 
conceptual understanding/problem solving.” Another four items were designed to determine the extent to 
which a teacher’s lesson had a procedural emphasis. For example, observers rated a lesson’s procedural 
emphasis using this item: “Students had opportunities to learn procedures (by teacher demonstration, class 
discussion, or some other means) before they practiced them.” Factor analysis of the LieCal observation 
data confirmed that the four procedural-emphasis items loaded on a single factor, as did the four 
conceptual-emphasis items. Since students changed their classrooms and teachers as they moved from 
grade 6 to grade 7 and from grade 7 to grade 8, each student could have a different value each year for 
three years, but all students in the same classroom at each grade had the same value. 


Quantitative Data Analysis 


To examine student growth within each school year while controlling for multiple factors such as 
gender, ethnicity, and classroom conceptual and procedural emphases, we used hierarchical linear 
modeling (HLM). After unconditional models were fitted, two sets of conditional cross-sectional HLM 
analyses were conducted. The first set of models was composed of cross-sectional hierarchical linear 
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models that included student-level variables and a curriculum variable. These models used four different 
student achievement measures: open-ended, translation, computation, and equation solving. Each HLM 
model used data from one of the four dependent achievement measures in one of three middle grades, 
together with an independent prior achievement measure, namely the results of the state mathematics 
testing in the fall of the corresponding year. So, each model examined a single type of learning within a 
specific grade level. Since we had four achievement measures at each of three grade levels, there were 12 
cross-sectional models in this first group. 

The next set of models built on the first group of models by adding two classroom-level variables: the 
conceptual emphasis of the classroom and the procedural emphasis of the classroom. These cross-sectional 
HLM models were of the following form: 


Level-1 Model 

Yj = Po; + pi(Prior Achievementi;, - Xi) + po(Gender;;, - X2) 
+ p3(African American, - X3) + p4(Hispanic;,, - Xa) 
+ ps(Other Ethnicity; - Xs) + Tix 


Level-2 Model 
Poj = boo + bo, CMP + by,Conceptual Emphasis; + b);Procedural Emphasis; + 19; 


Interactions between conceptual emphasis, procedural emphasis, and curricula were tested, but found to be 
not significant. 
Results 


We present the results of our analysis in two parts. First, we report on the cross-sectional HLM models 
that included student-level and curriculum variables. Then, we examine the impact of including the 
classroom-level conceptual and procedural emphasis variables in the models. 


Student-Level and Curriculum Cross-sectional HLM Models 


Table 1 shows the standardized results from an examination of the performance of African-American 
and Hispanic students relative to White students, when controlling for prior achievement, gender, and 
curriculum (but not conceptual and procedural classroom emphases). 


Table 1: Effect of Ethnicity on Mathematics Achievement 


Grade 6 Grade 7 Grade 8 
African Hispanic African Hispanic African Hispanic 
American American American 
Open-ended -0.50*** -0.21* -0.26** -0.22* -0.28*** -0.13* 
Translation -0.24** -- -- -- -- -- 
Computation -0.37*** -0.22* -- -- -- -- 
Equation -0.35** -0.23* -0.24** -0.22* -- -- 


solving 


* p< .05. **p<.01. *** p<.001. 
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In the sixth grade, an achievement gap was seen between African American students and White 
students on all four student achievement measures, and between Hispanic students and White students on 
the open-ended, computation, and equation solving measures. The gaps on the open-ended and equation 
solving measures remained in the seventh grade for both groups. However, performance on the 
computation and translation measures had equalized across the groups. In the eighth grade, the only gap 
that remained was on the open-ended items. The overall trend was a gradual decline or elimination of the 
achievement gap among the ethnic groups. 

To better understand if using the CMP curriculum would reduce achievement gaps, we conducted 
separate parallel analyses for CMP and non-CMP students. The results are shown in Table 2. In the 
analysis of the combined CMP and non-CMP student sample, achievement gaps for the translation and 
computation measures occurred only in the 6" grade: White students outperformed African American 
students on both measures, and White students outperformed Hispanic students on computation. However, 
in the analyses of the separate student samples, we found that although all three of these gaps appeared for 
the non-CMP students, the only achievement gap for the 6" grade CMP students was in computation. In 
grades 7 and 8, the performance parity on computation and translation items observed in the combined 
sample of students was mostly preserved in the separate analyses, except for the appearance of a gap 
between CMP 8th grade African American students and White students on computation items. 

Mirroring the results from the combined sample, White students outperformed African American 
students on open-ended items across all three grades regardless of curriculum. For students using CMP, 
White students also outperformed Hispanic students on these items in Grades 7 and 8. For non-CMP 
Hispanic students, however, there were no parallel achievement gaps. For the equation solving items in the 
combined analysis, White students outperformed African-American and Hispanic students in grades 6 and 
7, with no achievement gap in grade 8. These gaps were attributable to the CMP students; there were no 
achievement gaps found for equation solving items among the non-CMP students. For CMP students, 
White students outperformed African American students in all three grades, and White students 
outperformed Hispanic students in grades 7 and 8. 


Table 2: Effect of Ethnicity on Mathematics Achievement for CMP / Non-CMP Students 


Grade 6 Grade 7 Grade 8 
African Hispanic African Hispanic African Hispanic 
American American American 
Open-ended -0.40** / -- / -0.27** / -0.31**/ -0.36** / -0.28**/ 
-0.91 *** -- -0.23* -- -0.26** -- 
Translation -- / -- / -- / -- / -- / -- / 
-0.37* -- -- -- -- -- 
Computation -0.43** / -- / --/ -- / -0.22* / -- / 
-0.27** -0.35** -- -- -- -- 
Equation -0.35* / --/ -0.44** / -0.45**/ -0.23**/ -0.22*/ 


solving -- -- -- 2u = cs 


*p<.05. **p<.0l. *** p< .001. 


Student-Level, Classroom-Level and Curriculum HLM Models 


We built on the results of Table 1 with the addition of the conceptual emphasis and procedural 
emphasis classroom-level variables. Our goal in adding these variables to the analysis was to begin to 
probe the complexity that underlies conclusions we might otherwise draw from one-dimensional 
comparisons of students in different ethnic groups. With respect to the analysis of the combined CMP and 
non-CMP students, however, the addition of the classroom-level variables did not greatly perturb the 
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results save for the disappearance of the gap in Hispanic students’ performance on open-ended tasks in the 
8th grade. 

We again conducted parallel analyses for the CMP and non-CMP students, this time including the 
conceptual and procedural emphasis classroom-level variables. The results are presented in Table 3. For 
the CMP students, two achievement gaps were no longer statistically significant with the addition of the 
classroom variables: 8th grade African American students on computation items, and 8th grade Hispanic 
students on equation solving items. For non-CMP students, the performance gap of 6th grade African 
American students on translation and computation items ceased to be significant. 


Table 3: Effect of Ethnicity on Mathematics Achievement for CMP/Non-CMP Students 
Controlling for Conceptual and Procedural Emphases 


Grade 6 Grade 7 Grade 8 
African Hispanic African Hispanic African Hispanic 
American American American 

Open-ended -0.40** / - --/ -0.28** / - -0.30**/ -0.36** / - -0.29**/ -- 

0.90*** -0.33* 0.27* -- 0.23** 
Translation -- /-- -- /-- -- /-- -- /-- -- /-- --/-- 
Computation -0.37** / --/ -- /-- -- /-- -- /-- --/-- 

-- -0.33** 

Equation -0.35*/ -- /-- -0.45** / -- -0.44**/ -- -0.21** / -- -- /-- 


solving -- 


*p<.05. **p<.0l. *** p<.001. 


For the combined student groups, the performance of Hispanic students in the 8th grade was not 
significantly different from 8th grade White students for all four achievement measures. Similarly, the 
performances of 8th grade African American and White students were not significantly different except on 
the open-ended items; there was no achievement gap between African American and White students in the 
8th grade on translation, computation, and equation solving items. When analyzed as separate groups, the 
CMP and non-CMP students of color generally showed achievement gaps on open-ended items compared 
to White students using the same curriculum. Within the CMP student group, there were also achievement 
gaps for African American students on equation solving items. 


Discussion 


In examining how Standards-based curricula such as CMP affect the mathematics learning of students 
of color, it is important to use nuanced analyses to look beyond one-dimensional comparisons (Lubienski, 
2008). The longitudinal growth curve analysis of the LieCal data provided mixed conclusions regarding 
the use of the CMP curriculum with students of color (Cai et al., 2011). Though, over the course of the 
middle grades, African American and Hispanic students had growth rates similar to students not in their 
ethnic groups on the open-ended, translation, and equation solving measures, African American students 
had a smaller growth rate on the computation measure. The cross-sectional HLM analysis in this paper 
provides detail not captured in the longitudinal analysis. 

Overall, the results of the cross-sectional analysis show a trend of decreasing gaps in achievement. 
Whereas Hispanic and African American students score significantly lower than White students on most or 
all of the measures at the end of 6th grade, by the end of 8th grade, only the open-ended measure still 
reflects a gap. Moreover, when classroom conceptual and procedural emphasis is taken into account, the 
only difference that remains at the end of 8th grade is in African-American students’ performance on the 
open-ended tasks. Despite the slower growth rate in African American students’ performance on 
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computation tasks that was identified in the longitudinal analysis, the effect seems to be largely limited to 
the 6th grade. 

When the cross-sectional analysis is limited to the CMP students, the open-ended measure reflects a 
persistent gap between White students and students of color. Similarly, for African American students in 
the CMP group, equation solving remains an area of challenge throughout the middle grades. Even when 
classroom conceptual and procedural emphasis variables are included, these gaps remain. Indeed, these 
performance gaps in the CMP analysis do not decrease with grade level, as many of the other performance 
gaps do. Thus, despite the fact that the longitudinal analysis showed comparable growth curves for White 
students and students of color on the open-ended measure, the cross-sectional analysis suggests that there 
may be opportunities within the CMP curriculum for developing open-ended problem-solving skills that 
are being differentially accessed by students of different ethnic backgrounds. 

It is interesting to note how the influence of classroom emphasis variables played out differently for 
different student groups. For example, the profile of Hispanic CMP students’ equation solving 
performance was somewhat different from the African American students’. For Hispanics, the negative 
CMP effect was limited to the 7th grade. Classroom conceptual and procedural emphases, not curriculum, 
appear to account for Hispanic student performance differences in the 8th grade. Moreover, the reverse 
appears to be the case with respect to the translation and computation measures in the 6th grade. When 
controlling for classroom emphasis, there was no longer an achievement gap for African American 
students. This difference in the effects of classroom emphasis on Hispanic and African American students 
merits exploration. 

In conclusion, the longitudinal and cross-sectional analyses continue to paint a mixed picture of the 
effects of the CMP curriculum for students of color. By grade 8, most performance differences on the 
measures in this study were no longer significant. Though African-American students’ computation skills 
appeared to grow more slowly across grades 6 through 8, the effect of this difference seems to have been 
primarily limited to grade 6. However, the persistent gaps between African American students and White 
students on the open-ended and equation solving measures, even when classroom emphases are taken into 
account, invite further investigation. 
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